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ABSTRACT: This study evaluated the efficacy of the Preschool First Step (PFS) to Success early 
intervention for children at risk for attention deficit hyperactivity disorder (ADHD). PFS is a targeted 
intervention for children 3-5 years old with externalizing behavior problems and addresses secondary 
prevention goals and objectives. As part of a larger multisite, randomized controlled trial, the efficacy of 
the PFS program was evaluated on a subsample of 45 children who also had elevated comorbid 
ADHD symptoms as rated by parents and teachers. The PFS program was found to produce 
significantly higher social skills, and significantly fewer behavior problems across a variety of teacher- 
and parent-reported measures at postintervention. Effect sizes for teacher-reported effects were large 
across a variety of social competency indicators, including those specific to ADHD. Effect sizes for 
parent-reported social skills and problem behaviors were medium. Although not specifically designed 
for preschoolers at risk for comorbid ADHD, this generic behavioral intervention appeared to be 


successful for this population. Implications and limitations of the study are discussed. 


HE Most children with attention-deficit/hyper- 
activity disorder (ADHD) tend to have a pre- 
school onset of at least some of their symptoms. 
Recent surveillance research on a longitudinal 
birth cohort suggested that pediatric predictors 
of ADHD were evident by 17 months of age 
and that children with the highest levels of 
hyperactive behaviors at 17 months still had 
very stable high levels of hyperactive behaviors 
at age 6 (Galéra et al., 2011). Using six longitu- 
dinal data sets in a meta-analysis, Duncan and 
colleagues (2007) found that the strongest pre- 
dictors of school readiness and later achieve- 
ment were much more likely to be preschool 
attention skills than externalizing or internaliz- 
ing disorders. They also concluded that changes 
in attention skills during the preschool years 
were especially strong predictors. In another 
longitudinal study of real-life dyadic friendships 
of young children with ADHD and comparison 
children over time, Normand et al. (2013) 
reported significantly poorer friendship quality, 
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more friendship conflict, and less friendship satis- 
faction for children with ADHD over the same 
period. Thus, early interventions that target 
children at risk for ADHD in preschool appear 
to be a potential avenue for improving both 
educational attainment and social functioning. 

Evidence-based treatment for ADHD usually 
involves stimulant medication and/or behavioral 
programs, such as behavioral parent training 
or teacher consultation (Konopasek & Forness, 
2014). Charach and colleagues (2013) recently 
conducted a comparative effectiveness review 
of 55 studies on preschoolers with ADHD 
involving medication, behavioral intervention, 
or their combination. Behavioral interventions 
produced a mean effect size of 0.75 on child 
behaviors but only 0.55 on parent skill levels. 
The findings on stimulant medication were 
more equivocal because many of the studies 
were quite small. An exception was the Pre- 
school ADHD Treatment Study (PATS) that 
involved 304 preschoolers across six different 
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sites (Greenhill et al., 2006). Core symptoms of 
ADHD improved significantly with stimulant 
medication treatment over the initial phase of 
the study, as rated by both teachers and parents, 
but long-term follow-up suggested that most 
children with ADHD retained their diagnosis 
6 years later and that this was especially true 
of children with comorbid conduct or opposi- 
tional defiant disorder (Riddle et al., 2013). 
Authors suggested that more emphasis on class- 
room intervention, rather than just parent train- 
ing, might have improved outcomes. It was also 
noted that preschool children were twice as 
sensitive, compared to elementary school chil- 
dren, to side effects of methylphenidate even 
though the dose was kept relatively low (March, 
2011). Thus, behavioral interventions remain 
the treatment of choice for most preschoolers 
with ADHD. For young children, however, 
most classroom-based early interventions do 
not target specific diagnoses, such as ADHD, 
but focus on broader goals to improve social 
skills or academic readiness and de- 
crease risk behaviors (Maag & Katsiyannis, 
2010; Rajwan, Chako, & Moeller, 2012). Such 
generic interventions typically cast a wider net 
and focus on clusters of age-appropriate risk 
behaviors rather than on one specific disorder 
at a time, the usual target of most mental health 
treatments (Forness, 2012). 

Dunlap and Fox (2014) described half a 
dozen such generic behavioral interventions 
that have met relatively strict standards for 
evidence-based practice for preschool children 
at risk for behavioral problems in the classroom 
setting. One of these is First Step to Success 
(Walker et al., 1998). First Step is designed to 
be used in the secondary tier of a multitiered 
model of prevention; the intervention forges 
a home and school partnership in which the 
teacher, the child’s parents, and the First Step 
behavioral coach work together in teaching 
the target child school success skills (Walker 
et al., 2014). Although originally developed 
for application with a range of at risk externaliz- 
ing behaviors, a previous investigation of a 
subset of children at risk for ADHD in a larger 
randomized controlled trial of elementary chil- 
dren showed considerable promise (Seeley 
et al., 2009), but such findings may not necessa- 
rily generalize to younger preschool-age chil- 
dren at risk for ADHD. A preschool adaptation 
of the First Step intervention has recently been 
developed (Feil, Small et al., 2009; Frey et al., 
2013). In this study, we attempted to assess 
the extent to which such a_ broad-based 
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intervention might also potentially benefit a 
subsample of preschool children at risk for 
ADHD who were identified retrospectively as 
being at risk for ADHD in a large randomized 
controlled trial of Preschool First Step (PFS; 
Feil et al., 2014). 

Such subgroup analyses are somewhat con- 
troversial but are frequently used when there 
is compelling evidence that a subgroup may 
be at particular risk and thus might respond 
differentially to a generic intervention (Sun, 
loannidis, Agoritsas, Alba, & Guyatt, 2014). 
There has long been evidence that children 
with externalizing disorders, the intended target 
for First Step intervention, are at greater risk 
when they have comorbid ADHD (Gresham, 
MacMillan, Bocian, Ward, & Forness, 1988). 
Recent evidence has suggested that this comor- 
bidity is also a factor in long-term nonresponse 
of preschoolers to stimulant medication treat- 
ment as well (Vitiello et al., 2015). 

Our objectives for this subgroup analysis 
were (a) to determine whether PFS significantly 
improves the general behavioral symptoms and 
social functioning of preschoolers with externa- 
lizing disorders who are also at risk for ADHD, 
and (b) to determine if this intervention likewise 
significantly improves their behavioral symp- 
toms and social functioning specific to ADHD. 
A secondary objective was to examine the clin- 
ical impact of PFS on preschoolers with this 
comorbidity. 


Method 


Participants 


Participants for this study included a 
subsample of 45 of the original 126 child- 
parent-teacher triads from Head Start and pre- 
school programs in Oregon, Kentucky, and 
Indiana who were participants in the original 
PFS randomized control trial (Feil et al., 2014). 
Project staff recruited these triads across three 
cohorts between 2009 and 2012. Random 
assignment to the PFS intervention condition 
or usual-care control condition occurred at the 
classroom level. For each classroom, one child 
who exhibited elevated externalizing behavior 
based on teacher report was recruited and con- 
sented to participate, as described below. 
Note that although the classroom was the unit 
of randomization in our original randomized 
control trial, this may be misleading in that tea- 
chers screened their entire classroom using a 
multiple gating procedure (described below) 
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to select one target child for intervention. 
Thereafter the PFS intervention focused on just 
that one child-parent-teacher triad per 
classroom. 


Procedures for the Original Full Sample 


More detail on these procedures is avail- 
able in Feil et al. (2014), but a brief overview 
is presented here. We obtained Institutional 
Review Board approval for the original study 
and then recruited and consented participating 
teachers and parents. Teachers used an adapted 
version of the Early Screening Project (ESP; 
Walker, Severson, & Feil, 1995) to identify chil- 
dren in their classroom exhibiting externalizing 
behavior. The ESP is a gated screening proce- 
dure in which teachers nominated and rank- 
ordered five children in their classroom who 
matched most closely a description of external- 
izing behavior. For each child identified, 
teachers completed three Stage 2 rating scales 
from the ESP: the Adaptive Behavior Index, 
Maladaptive Behavior Index, and Aggressive 
Behavior Scale. Project staff scored the rating 
scales, converted raw scores to severity scores, 
and rank-ordered the five children within each 
classroom to identify the highest ranked child. 
If we could not recruit the highest ranked child, 
we attempted to recruit the next ranked child in 
the classroom. The process continued until we 
obtained parent consent for one eligible child 
or the parents of all eligible children had 
declined participation. Project staff collected 
baseline data and then randomized each child- 
parent-teacher triad to a PFS intervention condi- 
tion or usual-care control condition. 

The 126 children in the full trial had a 
mean age of 4 (SD = 0.4) and were predomi- 
nantly male (65%). Most children were African 
American (31%) or Caucasian (44%). Nearly 
all teachers were female (99%) and most were 
either African American (18%) or Caucasian 
(72%). Teachers had taught for approximately 
14 years, and 22% had a high school diploma, 
33% an associate’s degree, 23% a bachelor’s 
degree, and 22% a master’s degree or higher. 

Teachers in classrooms randomized to the 
intervention condition received (a) 4 hr of train- 
ing in general education classroom manage- 
ment and positive behavior support; (b) 4 hr 
of training in PFS (described in detail below); 
and (c) ongoing, one-on-one consultation and 
support from a behavioral coach who worked 
with the teacher to implement the program. 
Teachers randomized to the usual-care control 
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condition received only the same 4 hr of 
training in general education classroom man- 
agement strategies. For parents randomized to 
the intervention condition, a behavioral coach 
worked with them for 6 weeks in 1-hr sessions 
to promote school success skills via reading, 
discussion, role play, and demonstrations. 
Coaches were employees of Oregon Research 
Institute or University of Louisville (eight 
coaches at each site) and held a bachelor’s 
degree or higher. Coaches attended a 2-day 
intensive training session on PFS implementa- 
tion. Lead implementers met weekly with 
coaches to discuss and troubleshoot cases and 
were closely monitored via frequent fidelity 
checks during implementation. Postinterven- 
tion data were collected roughly 4 months later, 
at completion of the intervention, and were 
yoked in timing to usual-care control triads for 
equivalency between groups. 


Procedures for ADHD subsample 


Although we did not report ADHD sympto- 
mology or functioning in our original main 
effects study (Feil et al., 2014), we had none- 
theless collected data on such measures in 
anticipation of a subgroup or other moderator 
analysis. Consistent with recommendations by 
Rothwell (2005) for subgroup construction and 
analysis, we used parent and teacher report on 
the Conners’ ADHD Scales (CADS; Conners, 
1999) to identify retrospectively the subsample 
of children at risk for ADHD in this study. The 
CADS scale has excellent evidence of reliability 
and validity and includes 18 items assessing 
the presence of ADHD symptomatology in the 
last month. Parents and teachers rate items on 
a 4-point rating scale ranging from not true at 
all to very much true. We used CADS normative 
data to identify the ADHD sample of children, 
with both parent-reported and teacher-reported 
ADHD total scores needing to be 1 SD or 
more above the mean to meet inclusion criteria. 
Applying this cutoff, 45 of the 126 children par- 
ticipating in the efficacy trial (35%) met criteria 
for being at risk of developing ADHD. We com- 
pared the ADHD sample identified for this 
report (n = 45) to the remaining nonidentified 
children from the sample (n = 81) on baseline 
outcome measures. As expected and as 
depicted in Table 1, the identified and noniden- 
tified children differed significantly on 
five of 10 baseline measures and approached 
significance (p < .10) on three other baseline 
measures. 
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TABLE 1 
Baseline Means and Standard Deviations by Identified and Nonidentified Samples for Outcome 
Measures 


Identified Sample (n = 45) —_ Nonidentified Sample (n = 81) Test Statistic 


Variable M (SD) 

Symptoms 

SSiS Hyperactive/Inattentive 14.2 (3.0) 

C-TRF ODD 7.9 (3.3) 

ESP MBI 30.4 (6.1) 

ESP ABS 22.4 (6.7) 
Functioning 

ESP ABI 21.6 (4.5) 

SSiS Cooperation 6.4 (2.1) 

SSiS Engagement 10.9 (4.0) 

SSiS Self-Control 6.6 (3.1) 
Home-based 

SSiS-PB 125.9 (12.7) 

SSiS-SS 88.1 (12.4) 


M (SD) t p Value 
12.3 (3.6) -2.93 .004 
6.6 (3.4) -1.93 .056 
28.7 (6.8) -1.44 152 
19.1 (5.8) —2.88 .005 
23.2 (4.9) 1.79 .076 
7.4 (2.8) Pe 025 
10.7 (3.6) —0.24 .814 
7.7 (3.5) 1.66 .099 
113.3 (17.2) —4.28 <.001 
94.9 (13.6) 2.73 .007 


Note. SSiS = Social Skills Improvement System Rating Scales; C-TRF ODD = Caregiver-Teacher Report Form Oppositional Defi- 
ant Disorder; ESP MBI = Early Screening Project Maladaptive Behavior Index; ESP ABS = Early Screening Project Aggressive 
Behavior Scale; ESP ABI = Early Screening Project Adaptive Behavior Index; SSiS-PB = Social Skills Improvement System Rating 
Scales Problem Behaviors; SSiS-SS = Social Skills Improvement System Rating Scales - Social Skills. 


Of the 45 children at risk for ADHD, 19 had 
been randomized to the usual-care condition 
and 26 children had been randomized to 
the PFS condition. To establish baseline equiva- 
lency between intervention and control condi- 
tions, we examined child, parent, and teacher 
baseline demographics and baseline outcome 
measures for the two conditions. There were 
no statistically significant differences between 
the two groups on basic child demographic 


and screening characteristics as reported in 
Table 2. Not shown in Table 2, the control 
and intervention conditions also did not differ 
on other examined demographic variables 
including age of participating parent (M[SD] = 
30.5[7.9] vs. 30.3[6.3]), percent of parents 
who were African American (37% vs. 35%), or 
Caucasian (53% vs. 58%), percent of parents 
with a bachelor’s degree or higher (5% vs. 
7%), or the number of children in the 


TABLE 2 
Baseline Equivalence of Demographic Characteristics and Screening Measures 
Control/Usual Care _ Intervention Test P 
Variable Total (n = 45) (n = 19) (n = 26) Statistic Value 
Demographic characteristic 
Age M (SD) 4.1 (0.4) 4.1 (0.4) 4.1 (0.4) -0.20 -840 
Female (%) 16 (35.6) 6 (31.6) 10 (38.5) 0.23 634 
African American (%) 16 (35.6) 7 (36.8) 9 (34.6) 0.02 878 
Caucasian (%) 22 (48.9) 8 (42.1) 14 (53.8) 0.61 436 
IFSP (%) 12 (26.7) 4 (21.1) 8 (30.8) 0.53 467 
Screening measures 
ESP rank 0.36 833 
Ranked first (%) 36 (80.0) 16 (84.2) 20 (76.9) 
Ranked second (%) 6 (13.3) 2 (10.5) 4 (15.4) 
Ranked third (%) 3 (6.7) 1 (5.3) ZAP 7) 
Aggressive Behavior Scale M (SD) 23.4 (5.5) 23.6 (7.0) 23.2 (4.3) 0.26 795 
Adaptive Behavior Index M (SD) 20.8 (4.0) 21.3 (3.6) 20.5 (4.3) 0.66 514 
Maladaptive Behavior Index M (SD) 32.2 (5.3) 31.8 (5.9) 32.5 (5.0) —0.46 .646 


Note. Reported test statistics are t for continuous and x’ for dichotomous measures. IFSP is percent of children having an indivi- 
dualized family services plan, indicating receipt of special education services. ESP = Early Screening Project. 
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household (M[SD] = 2.4[1.5] vs. 2.4[0.9]). 
Across conditions, all teachers were female. 
The majority of teachers in the control and 
intervention groups was Caucasian (74% and 
77%, respectively). A disproportionate percen- 
tage of teachers in the control condition had 
reportedly attained a bachelor’s degree or 
higher (63%) as compared to the intervention 
condition (31%; x? = 4.66, p = .030). Although 
nonsignificant, teachers in usual-care condition 
reported teaching for more years (M[SD] = 
18.1[9.5]) than teachers in intervention condi- 
tion (M[SD] = 13.6[8.6]; t[42] = 1.63, p = .11). 


Preschool First Step to Success 
Intervention 


The PFS intervention included a day-long 
workshop for teachers in the universal prin- 
ciples of classroom management (Sprague & 
Golly, 2013) and also training in the PFS inter- 
vention (Walker et al., 1998). A behavioral 
coach was then assigned to each participating 
teacher and was available for one-on-one 
consultation in his or her respective classroom 
during instructional hours. PFS is described 
very briefly below (for more detailed descrip- 
tions, see Feil et al., 2014; Frey et al., 2013). 


Classroom-Based Component 


Implementation of the classroom compo- 
nent had three phases of 10 days each: (a) the 
coach phase, (b) the teacher phase, and (c) 
maintenance. The PFS program involves pro- 
viding feedback initially from the coach and 
then the teachers, using a green and red card 
—green for positive classroom behavior and 
red for off-task behavior. The participating 
child receives points and praise for engaging 
in appropriate classroom behavior (e.g., follow- 
ing classroom rules, cooperating, sharing, 
sitting quietly during circle time). Group depen- 
dent contingencies are used not only to moti- 
vate the participating child but also to provide 
incentives for the child’s classmates to invest 
in his/her success; individual contingencies 
through a home-school reward system provide 
additional incentives to support the child’s 
mastery of school success skills. 


Home-Based Component 


Over this same period, caregivers meet for 
six home visitation sessions with the First Step 
coach to learn how to teach school success 
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skills via reading, discussion, role play, and 
demonstrations. Specific skills taught are com- 
munication and sharing, cooperation, limit set- 
ting, problem solving, friendship making, and 
self-confidence. Parents are also provided with 
a manual containing all the information and 
accompanying materials needed to implement 
this home component. 


School-Based Outcome Measures 


As part of the larger assessment battery for 
the full study, teachers and parents completed 
the Social Skills Improvement System Rating 
Scales (SSiS; Gresham & Elliott, 2008). Teachers 
completed three ESP Stage 2 rating scales (Feil, 
Severson, & Walker, 1998) and the Achenbach 
Caregiver-Teacher Report Form DSM (Diagnos- 
tic and Statistical Manual of Mental Disorders)- 
oriented subscale for Oppositional Defiant 
Disorder (ODD; Achenbach & Rescorla, 
2001). We used these measures divided into 
two domains for this study as described below: 
(a) ADHD and disruptive behavior symptoms, 
and (b) social functioning. 


ADHD and Disruptive Behavior Symptoms 


For this study, symptom scales included 
a measure specific to ADHD hyperactive 
and inattentive behavior, along with a general 
measure of maladaptive behavior, a general 
measure of aggressive behavior, and one mea- 
sure of ODD symptoms (mean intercorrelation 
= .62). To examine ADHD-specific variables, 
we used inattentive and hyperactive behavior 
scores on the SSiS Hyperactivity/Inattention 
subscale. The seven-item scale (a = .76) exam- 
ines the teacher’s perceived frequency of 
hyperactive or inattentive behaviors such as fid- 
geting, impulsivity, distractibility, difficulty 
waiting for turns, and disrupting group activities 
(Gresham & Elliott, 2008). SSiS items are 
reported on a 4-point frequency scale (never, 
seldom, often, almost always). Scores can range 
from O to 21 with higher scores indicating 
higher levels of impairment. The ESP Maladap- 
tive Behavior Index is a nine-item scale 
(a = .81) assessing the child’s teacher-related 
and peer-to-peer maladaptive behavioral 
adjustments. The ESP Aggressive Behavior 
Scale is a nine-item scale (« = .79) measuring 
the frequency of aggressive behavior. The 
Maladaptive Behavior Index and Aggressive 
Behavior Scale are rated on a 5-point frequency 
scale ranging from never to frequently. Raw 
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scale scores (ranging from 9 to 45) were com- 
puted for each measure. Higher scores indicate 
higher levels of problem behavior. The Care- 
giver-Teacher Report Form ODD measure is a 
seven-item (a = .82) DSM-oriented subscale 
assessing ODD symptoms. Caregiver-Teacher 
Report Form items are rated on a 3-point scale 
from not true to very true. Raw scores range 
from 0 to 14 with higher scores indicating 
higher levels of ODD symptomatology. 


Social Functioning 


Measures of social functioning included 
ADHD-related measures of cooperation, en- 
gagement, and self-control, as well as a general 
measure of adaptive behavior (mean intercorre- 
lation = .36). The SSiS cooperation subscale is 
a six-item scale (a = .79) assessing behaviors 
such as following rules and directions, complet- 
ing tasks, and participating appropriately. The 
SSiS engagement subscale is a seven-item mea- 
sure (« = .83) assessing the child’s ability to 
join activities, make friends, and interact with 
others. The SSiS self-control subscale consists 
of seven items (a = .76) assessing the child’s 
ability to resolve disagreements, accept criti- 
cism, and respond calmly to teasing or bullying. 
These three SSiS subscales are rated on the 
same 4-point frequency scale described above 
and were used as measures of particular con- 
cern in functioning of children with ADHD. 
Higher scores on the three scales indicate 
higher levels of school-based functioning. The 
ESP Adaptive Behavior Index is a more general 
eight-item scale (« = .77) assessing teacher- 
related and peer-to-peer adaptive behavioral 
adjustments. Adaptive Behavior Index items 
are scored on a 5-point frequency scale. Raw 
scores range from 8 to 40 with higher scores 
indicating higher levels of adaptive functioning. 


Home-Based Outcome Measures 


Home-based outcomes included parent 
report for the SSiS social skills and problem 
behavior scales. The social skills scale targets 
learned behaviors that promote pro-social inter- 
actions with adults and peers (Gresham & Elliott, 
2008). It includes 46 items (« = .95) assessing 
the child’s development of social skills pertain- 
ing to day-to-day interaction and activities in 
the home setting. The 33-item (a = .92) prob- 
lem behavior scale assesses internalizing and 
externalizing behaviors that may interfere with 
the child’s social functioning. Informants report 


100 / February 2016 


their perceived frequency of the child’s prob- 
lem behaviors on a 4-point scale. 


Process Measures 


We collected PFS coach and teacher imple- 
mentation fidelity data related to the school 
component and data to identify dosage, parent 
fidelity, and parent compliance. We also col- 
lected indicators of teacher and coach alliance, 
as well as parent and teacher satisfaction data 
for participants randomized to the intervention 
condition. The implementation fidelity check- 
list assessed the adherence and quality of class- 
room implementation and was collected once 
during the coach phase and twice during the 
teacher phase with an interrater reliability of 
.82. The classroom monitoring form was used 
to record the target child’s compliance during 
the classroom component such as daily points 
earned, whether the daily criterion was met, or 
if the program day was repeated because the 
child did not meet the daily reward criterion. 
Classroom dosage is the proportion of program 
days out of 30 completed. The home monitor- 
ing form was used to compute dosage, parent 
fidelity, and parent compliance for the home 
component of First Step and is completed by 
the First Step coach after each session. The alli- 
ance survey was completed by the coach and 
teacher at postintervention to assess various 
aspects of their working relationship. Coeffi- 
cient alpha was .94 for the coach version and 
.95 for the teacher version. The satisfaction sur- 
vey was collected by teachers and parents at 
postintervention, and assessed the perception 
of training and support received, program 
usability, and program effectiveness, with 
alphas of .91 and .94, respectively. Further 
details on all of the above measures are avail- 
able in Sumi et al. (2013), Walker et al. 
(2009), and Woodbridge et al. (2014). 


Statistical Analysis 


For each outcome, we specified a linear 
regression model in Mplus 6.0 (Muthén & 
Muthén, 1998-2010). We utilized the full 
information maximum likelihood estimator to 
account for missing data. Full information 
maximum likelihood utilizes all available data 
to calculate unbiased parameter estimates 
and standard errors (Schafer & Graham, 2002). 
Each regression model included one covariate 
(i.e., the baseline value of the outcome) and a 
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dichotomous predictor for intervention condi- 
tion (1 = PFS intervention, 0 = control). 

We applied the Benjamini-Hochberg cor- 
rection (B-H; Benjamini & Hochberg, 1995) to 
correct for multiple comparisons. To calculate 
a B-H correction, statistically significant out- 
comes are ranked in ascending order based on 
p value and a new critical cutoff for each is cal- 
culated using the formula p,’ = .05x/n, where x 
equals the rank for each outcome and n equals 
the number of ranked outcomes (What Works 
Clearinghouse [WWC], 2014). Applying this 
formula to the 10 outcomes reported in this 
manuscript, the first outcome would still be 
significant at p < .005 and the 10th outcome 
would still be significant at p < .05. 

We report Hedges’ g as a measure of effect 
size and the WWC (2011) improvement index 
as a measure of practical significance. Hedges’ 
g is the difference between the mean outcome 
score of each group divided by the pooled 
within-group standard deviation (WWC, 2011). 
Effect sizes of 0.25, 0.5, and 0.8 are considered 
small, medium, and large effects, respectively. 

We utilized Jacobson and Truax’s (1991) 
Reliable Change Index (RCI) to identify clinically 
significant postintervention response. Change is 
based on a two-step criterion that accounts for 
magnitude of change, the RCI, and change in 
functioning or movement across a specified cutoff 
(Jacobson, Roberts, Berns, & McGlinchey, 1999). 
To assess change in functioning or, more speci- 
fically, reduction in ADHD symptomatology, 
we set a cutoff at 1 SD above the normative 
mean (T = 59). We then classified children as 
responded if at postintervention they moved 
into the normative range on the CADS screen- 
ing measures and had a RCI less than —1.96; 
b) improved if they did not move into the nor- 
mative range but had an RCI less than —1.9; 
c) unchanged if they did not meet the RCI 
criterion (i.e, RCI > —1.96 and < 1.96); or 
d) deteriorated if the RCI criterion worsened 
i.e., RCI > 1.96). See Small et al. (2015) for 
more detail. 


fe) 


Results 
Attrition and Missing Data 


Of the 45 children in the ADHD sample, 
baseline questionnaire data were available for 
all teachers and parents. At postintervention, we 
obtained questionnaire data from 100% of tea- 
chers and 89% of parents. Missing item-level 
data precluded subscale scoring for two 
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additional cases. One child was missing parent- 
reported data for one of two outcomes at baseline 
and another child was missing teacher-reported 
data for four of eight outcomes. We utilized Lit- 
tle’s Missing Completely At-Random test (Little, 
1988) to examine the assumption that data were 
missing completely at random. The test was non- 
significant (y* = 48.72, df = 1, p = .641), suggest- 
ing data were indeed missing completely at 
random. 


Process Measures 


The summary of descriptive statistics for 
child, parent, and teacher process measures 
were as follows. Coaches adhered to 96%, and 
teachers to 94%, of observed core program 
components. Classroom implementation qual- 
ity was excellent for coaches (.92; range = 
.77-1.00) and acceptable for teachers 
(.78; range = .41-1.00). Children in the ADHD 
sample received 97% (range = 67%-100%) of 
program days. Families received 87% (range = 
17%-100%) of home sessions. Child compli- 
ance was excellent (.89), but parent compliance 
(.54) was low and parent fidelity (.78) was mod- 
erate. Informants rated highly their working 
relationship with one another. Mean coach rat- 
ings were 4.3 on a 5-point scale and mean 
teacher ratings were 4.9. Teachers and parents 
also reported high levels of program satisfaction 
(4.4 and 4.3, respectively, on a 5-point scale). 


Posttest Differences on Outcome 
Measures 


Table 3 summarizes results from the covari- 
ate-adjusted regression models. Even after a 
B-H_ correction, children who received the 
PFS intervention had statistically significant 
improvements from baseline to postintervention 
on all teacher-reported and parent-reported out- 
comes. For teacher-reported reductions in symp- 
toms, Hedges’ g effect sizes ranged from —1.02 
to -—1.14. For teacher-reported improvements 
in functioning, effect sizes ranged from 0.80 to 
1.40. Hedges’ g effect sizes for parent-reported 
outcomes were in the medium range for reduction 
in symptoms (—0.77) and improvements in 
functioning (0.65) in the home setting. 


Responder Analysis 


Based on teacher report, seven children in 
the intervention condition responded (27%), 
seven children improved (27%), and 12 child- 
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ren remained unchanged (46%). In compari- 
son, one child in the control group responded 
(5%), one child improved (5%), 15 remained 
unchanged (79%), and two children in the con- 
trol group deteriorated (11%). Overall, 14 inter- 
vention children (54%) either responded or 
improved after participating in the PFS inter- 
vention compared to two children (11%) in 
the control condition (y? = 8.99, p = .002, 
OR[95% Cl] = 9.9[1.9, 51.9]). The seven inter- 
vention children classified as responded had 
on average a 2 SD improvement in postinter- 
vention CADS scores (M[SD] = —20.7[4.6]). 
For the seven intervention children classified 
as improved, postintervention CADS_ scores 
decreased by more than 1.5 SDs on average 
(M[SD] = —16.3[5.2]). Eight of the 12 children 
classified as unchanged improved by more 
than a .5 SD (M[SD] = —5.9[3.1]) but did not 
demonstrate clinically meaningful improvement. 
Scores for the remaining four children classified 
as unchanged remained stable from baseline to 
post intervention (M[SD] = 0.8[1.0]). 

Based on parent report, 10 children in 
the intervention condition responded (39%), 
six children improved (23%), and six remained 
unchanged (23%). The remaining four children 
did not have parent-reported CADS data at 
postintervention. In comparison, two children 
in the control group (11%) responded, two chil- 
dren improved (11%), 11 remained unchanged 
(58%), and two deteriorated (11%). We did not 
have parent-reported CADS data at postinter- 
vention for the remaining two children in the 
control condition. Overall, 16 children (62%) 
in the intervention condition responded or im- 
proved based on parent report, whereas only 
four children (21%) in the control condition 
responded or improved (y? = 9.29, p = .002, 
OR[95% Cl] = 8.7[2.0, 37.4]). Scores for the 
10 intervention children classified as responded 
decreased, on average, by more than 1.5 
SDs (M[SD] = -17.9[10.0]). Scores for the six 
children classified as improved decreased by 
almost 1.5 SDs (M[SD] = —13.3[5.3]). 


Discussion 


This study suggests that PFS seems parti- 
cularly effective with preschool-age children 
with externalizing behavior who are also exhi- 
biting elevated comorbid ADHD symptoms. 
The PFS intervention was implemented with 
relatively high levels of fidelity and program 
compliance for this subsample at risk for ADHD. 
These preschoolers, furthermore, showed highly 
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significant improvement on a broad range of 
teacher- and parent-reported outcome mea- 
sures, with eight of 10 effect sizes in the large 
range. Such effect sizes were also higher rela- 
tive to those found in the original study in 
which only one of seven outcomes was large 
(Feil et al., 2014). Additionally, although PFS 
was not designed to target specific disorders 
such as ADHD, it nonetheless appears to im- 
prove both the specific symptoms and social 
functioning associated with this disorder, and 
its effect sizes also compare favorably with 
those for ADHD-specific interventions (Charach 
et al., 2013; Comer, Chow, Chan, Cooper-Vince, 
& Wilson, 2013). 

This study has some noteworthy limitations 
as well. First, our interest in examining the 
potential efficacy of the PFS for children at 
risk for ADHD did not begin with an a priori 
hypothesis but was based on a relatively small 
subsample selected after the fact. Such sub- 
sample analyses can thus only be considered 
valid as hypothesis-generating studies, and evi- 
dence-based practice recommendations must 
await further confirmation from a priori ran- 
domized clinical trials (Rothwell, 2005; Sun 
et al., 2014). Second, our participants in this 
subsample can only be described as “at risk.” 
Our results might have been different had we 
selected preschoolers on a more thorough diag- 
nostic assessment for ADHD involving more 
than one symptom-rating scale and concomi- 
tant measures of functional deficits or adminis- 
tration of a structured diagnostic interview. 
Third, our outcome measures were dependent 
on unblinded ratings by participants without 
further confirmation by direct observation or 
independent ratings. We might also note here 
the relatively more modest fidelity and compli- 
ance results from parents in this study, as well 
as lower home-base outcomes, suggesting a 
failure to fully engage and utilize families. For 
this reason, we have recently competed a pilot 
study (Frey et al., 2014) enhancing the PFS 
home component with motivational interview- 
ing strategies and a more intensive individ- 
ualized approach for each family, and these 
results have lead to a large randomized trial of 
First Step with these added modifications that 
is currently underway. 

In the context of this subgroup analysis, 
however, our study nonetheless had several 
strengths derived from the original sample 
including the use of a randomized design to 
control for threats to internal validity, measures 
that are well established in preschool 


February 2016 / 103 


intervention research, simultaneous replication 
across three different sites, a relatively ethni- 
cally diverse sample of participants, substantial 
evidence that our intervention and control con- 
ditions were relatively well matched at base- 
line, use of not only generic outcome 
measures but also of measures specific to 
ADHD symptoms and functioning, and a 
responder analysis. This last analysis suggests 
a substantial clinical response such that on tea- 
cher ratings, more than a quarter of children 
moved well into the normative range with treat- 
ment and another quarter improved quite signif- 
icantly (on average by 1.5 SDs). Such an 
improvement in preschool suggests a poten- 
tially favorable long-term outcome (Bussing 
et al., 2012). 

Although we cannot recommend PFS as 
either a comprehensive or specific treatment 
for preschool ADHD at this point, it should be 
noted that most children with ADHD may not 
be recognized as such until well into their 
elementary years given that ADHD may often 
be comorbid with other externalizing or even 
internalizing disorders, which complicates its 
diagnosis (Forness & Kavale, 2002). These chil- 
dren may nonetheless demonstrate significantly 
disruptive behavior well before school entry, 
which may prompt a preschool referral for ge- 
neric tier-two intervention. The selection of 
PFS as such an intervention might prove to be 
fortuitous in that several of the strategies 
employed in PFS seem quite consistent with 
approaches recommended in ADHD-specific 
interventions (Charach et al., 2013). 

As for implications for practice, it is impor- 
tant to note that more than a third of the 
126 preschoolers who were carefully screened 
for externalizing behavior in the original First 
Step trial also met criteria as being at risk for 
comorbid ADHD in this study. This is relatively 
consistent with epidemiologic studies of 
such comorbidity (Forness & Kavale, 2002). 
Thus, teachers should be aware that a sub- 
stantial number of children with oppositional, 
defiant, aggressive, or other externalizing 
symptoms may also be at risk for ADHD. 
In this study, 62% of preschoolers with com- 
orbid ADHD showed a significant clinical 
response to First Step (compared to 21% in 
the control group). This is impressive even 
in the context of response rates in more in- 
tensive ADHD interventions (Konopasek & 
Forness, 2014). 

Preschool teachers should note, however, 
that a number of children with externalizing 
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behaviors also seem to have an overlay of 
especially inattentive, impulsive, or hyperac- 
tive behaviors and may not always respond 
even to the best of such generic tier-two beha- 
vioral interventions. These nonresponders may 
indeed necessitate referral for a formal diagnos- 
tic workup for ADHD, possibly leading to more 
intensive tier-three intervention involving 
enhanced parent collaboration and classroom 
management strategies specific to ADHD. Tea- 
chers should also know that such evidence- 
based interventions for ADHD are character- 
ized by multimodal treatment involving not 
only such enhanced psychosocial interventions 
but also possible medication management for 
nonresponders by qualified physicians (DuPaul 
& Stoner, 2014). Thus the best of tier-two pre- 
school interventions such as First Step serve 
not only to improve behavioral and social out- 
comes of most children at risk but also provide 
more compelling evidence of further need 
for more intensive and focused _ interdisci- 
plinary treatment for nonresponders (Mitchell, 
Stormant, & Gage, 2011). 
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